[experiment] Try building Rust for AArch64 #13278
Conversation
|
https://github.com/msys2-arm/MINGW-packages/actions/runs/3147837656/jobs/5117738600 - this is running on a KVM virtual machine on a Raspberry Pi 4B 8GB, so whatever it does isn't going to happen fast 😁 - hopefully it produces something useful and not just a network issue |
|
It seems to have failed on dependencies, though I didn't see a download error. My guess is depending on the group |
|
https://github.com/msys2-arm/MINGW-packages/actions/runs/3153177053/jobs/5130433521 managed to get past the dependencies and sources download, hopefully will come up with a useful error this time. |
Wha? |
|
Shoot, uncommented 1 line too much. |
mati865
force-pushed
the
rust-aarch64
branch
from
9c27b2d
to
2f9c2bc
Compare
|
https://github.com/msys2-arm/MINGW-packages/actions/runs/3154365326/jobs/5132415331 |
|
https://github.com/msys2-arm/MINGW-packages/actions/runs/3155479209/jobs/5135142992 |
|
I have no idea why this error was thrown on AAarch64, it didn't do that for me when compiling x86_64 -> x86_64. |
|
https://github.com/msys2-arm/MINGW-packages/actions/runs/3160479614/jobs/5144981155 Probably needs mingw-w64-clang-x86_64-gcc-compat, or more configuration options to override the 'gcc' defaults. Strangely, the exact command referenced as not being found is not listed in the configure output |
|
|
|
https://github.com/msys2-arm/MINGW-packages/actions/runs/3161324190/jobs/5147172653 and more similar |
|
No idea why it worked entirely differently for me on local x86_64... |
|
https://github.com/msys2-arm/MINGW-packages/actions/runs/3162085756/jobs/5148624572 Seems to be building this time. Will update with outcome. |
|
|
This is not going to work. Thank you for bearing with me but I'll have to find different solution. |
|
What does it need? A real cross-compiler, like llvm-mingw or maybe a hack like #8762 (but the target llvm-config.exe would still be arm64, so maybe that would not work from an x86_64 host)? |
|
TBH I have no idea, somehow cross compiling under MSYS2 doesn't work at all. |
|
The wrapper hack works for cross-compiling arm64 python from x86_64 host, and I think I was able to use them to build something using cmake. I don't know anything about rust's build system though, that seems to be the main problem because it appears to need some sorting out between libraries for build and host/target. |
|
I screwed around with using my cross clang wrappers on x86_64 host, from clang64 MSYSTEM, and got further, but it did end up failing to run /clangarm64/bin/llvm-config.exe as expected. Will have to try on arm64 box. |
|
Hmm, it seems running in clangarm64 MSYSTEM makes it want to link arm64 libs into BUILD/x86_64 binaries... Will have to go from clang64 MSYSTEM |
|
The cross build finished. But, trying to use the resulting install to build itself with hanging there, with a rustc.exe using 1 core and not much ram (like 2MB) |
|
FWIW, packages uploaded to https://github.com/msys2-arm/MINGW-packages/releases/tag/rust-1.64.0-aarch64-1, that's tagged at the PKGBUILD used to make them. The branch where I injected your stage0 binaries is master...jeremyd2019:ced44d965443b70f7f39901dde92c403f9e9bf95 |
|
Uh oh, unwinding again! (this is from the rustc from the package I built, not the rustc from your binaries. I can check that one too but I bet it's similar) |
|
Oof, I'll inspect x86_64 later this week (maybe even tomorrow). |
|
similar for your rustc.exe |
|
I realize now I probably noticed this accidentally too... I wanted to see if your rustc.exe was completely broken or not, so I tried |
|
Some bad news: x86_64 binary works fine even with misspelled |
x86_64 try: |
|
If things are somewhat easy to test running without diving too deep into all of Rust, I can try to have a look at the unwinding issue. (Primarily, I could try to run the hanging executable in Wine on aarch64, where I can debug the unwinder and see what goes wrong. Ideally, if there's a linker map for the executable, or debug info/symbols/something, this can help pinpoint in which function the unwinding fails.) While there might be unwinding issues on i686 due to mismatches in how the |
|
Well, you can reproduce the hang by downloading and extracting https://github.com/mati865/rust-gnullvm-builds/releases/download/1.63.0/rustc-1.63.0-dev-aarch64-pc-windows-gnullvm.tar.xz and running You'd probably need @mati865 to get you any debug info though. |
I have some followup, it starts to make sense and if I'm right exceptions will be broken the same way on x86_64 or (very unlikely) they will work on both. Need to do few cross compilations though so it'd be the best for you to wait until I come back with more results. |
|
I have 2 AArch64 builds, when misspelling
Testing should be as simple as running |
|
|
Cannot even tell if it's good or bad. |
Can you describe what's different between the builds, to see how it correlates with the actual behaviour? Unfortunately, I don't think the libunwind sources help much for these kinds of issues, since most of the actual unwinding is done by the RtlUnwindEx call - that's why Wine is a great resource for debugging such cases, when you can poke into the actual unwinder. I can maybe try to have a look tomorrow. |
|
2 prints error then seg faults. windbg shows same as with 1 |
Basically Rust has to plug somehow into libgcc/libunwind to catch them, on x86_64 I think it's done by |
|
Uploaded build 3 that differs only in debuginfo and build 4 that has debuginfo and a small change that is unlikely to fix the problem. Unfortunately I couldn't figure out anything inspecting LLVM repo it looks like x86_64 Rusts code linking calling libunwind should also work with AArch64. |
|
3 crashes same as 1 and 2, since you mentioned debug info I caught the crash in windbg and created a minidump, and opened that in lldb (running rustc in lldb resulted in a different, less helpful backtrace) Will try 4 later |
|
4 is same as 3 |
Ok - how does it plug into 0 ends up hanging, unwinding without making any real progress. From So here it gets to 1 does seem to fare better; it first executes The tricky thing here is that when you get a crash during unwinding, this crash is converted into an NT exception which also gets passed via the same unwinding mechanism, even though you're in a place where unwinding seems to have broken down. So after this, the output has a lot more unwinding where it tries to handle the When looking at the big picture, further up in the log, we'd have this: So the target of the whole unwind is to unwind the stack up to (Here, TODO for next steps: Disassemble |
|
Gut feeling, it is trying to access an offset of 0x18 from a NULL object pointer. I was hoping the line numbers in the backtrace I was able to get with debug info would indicate which pointer is NULL (and then can reason about how it ended up NULL). Maybe the exception object got lost somehow. |
I presume that Rust on aarch64 in MSVC mode does work, so the most probable clue for what is broken would be this libunwind integration. Can you point towards the code where this is done for x86_64 today, and include what you have for aarch64 so far? In particular, what do the differences between binary 0 and 1 look like in the code? |
Here is C FFI declaration: https://github.com/rust-lang/rust/blob/91f128baf7704a477ab7c499143a160fb069b3ad/library/unwind/src/libunwind.rs#L272 It's called in https://github.com/rust-lang/rust/blob/91f128baf7704a477ab7c499143a160fb069b3ad/library/std/src/personality/gcc.rs#L222 with
I have changed few cfgs (sort of
I have missed 1 cfg in build 0 so that was using the same path as Linux with libunwind (instead of default Linux with libgcc). With either build 1 or 2 (should have written it down...) AArch64 should use the same functions with the same arguments as x86_64.
I don't really have a clue what happened here but it appears that unwinding caught the exception but something is corrupted? Relevant code: https://github.com/rust-lang/rust/blob/e1d7dec558d863fb76f98453088b36cb1a926d48/compiler/rustc_driver/src/lib.rs#L1160
Haven't checked myself but since it was added quite long time ago by Microsoft folks IIRC I guess it's fine. |
Thanks for these pointers - that was useful to look at. This looks very much similar to how it is hooked up in libcxxabi and similar.
When the unwinding finishes and it resumes execution, it picks up here: First it calls So here, x0/x1 are forwarded as return values from In short, this function reads out two values from At this point, I cloned the rust source and sat down to look at this in the form of the rust source. In https://github.com/rust-lang/rust/blob/91f128baf7704a477ab7c499143a160fb069b3ad/library/panic_unwind/src/lib.rs#L95, However, the two pieces of data read from At this point, I instrumented Wine's unwinder to let it look at the pieces of data I want to, before it returns the control over to the rust code to do the cleanup. When Wine has finished unwinding, the pointer to the Now something might clobber these fields during the unwind, but I instrumented Wine at the very start of the unwind too, when rust/libunwind hands control over to the windows unwinder (via At the start So whatever rust specific private data that rust set in For x86_64, it has got Thus: TL/DR: Bump |
|
So that is the answer for:
Great job and what a fantastic writeup! I have uploaded build 5 to the same directory (direct link here). |
I don't really blog, I just write long replies on github :P But thanks - it's nice to hear that it's not all wasted words :-)
This version does indeed seem to run correctly - |
|
Superseded by #13513 |
Can somebody with AArch64 hardware try to build it and report the error that will inevitably occur?